SVM kernal Indepth Intution and Implementation

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

x = np.linspace(-5.0,5.0,100)
y = np.sqrt(10**2 - x**2)
y = np.hstack([y,-y])
x = np.hstack([x,-x])

x1 = np.linspace(-5.0,5.0,100)
y1 = np.sqrt(5**2 - x1**2)
y1 = np.hstack([y1,-y1])
x1 = np.hstack([x1,-x1])

plt.scatter(y,x)
plt.scatter(y1,x1)

<matplotlib.collections.PathCollection at 0x7f94822b7850>

For this we can not draw the bestfit line ,so we use SVM kernel trick to do !

df1 =  pd.DataFrame(np.vstack([y,x]).T,columns=['X1','X2'])
df1['Y'] = 0
df2 =  pd.DataFrame(np.vstack([y1,x1]).T,columns=['X1','X2'])
df2['Y']=1
df = df1.append(df2)
df.head()

	X1	X2	Y
0	8.660254	-5.00000	0
1	8.717792	-4.89899	0
2	8.773790	-4.79798	0
3	8.828277	-4.69697	0
4	8.881281	-4.59596	0

Dividing the Independent and Dependent Data

X = df.iloc[:,:2] #x1,x2
y = df.Y #Y -> 0,1

Using train_test_split to train and test

from sklearn.model_selection import train_test_split
X_train, X_test , y_train ,y_test = train_test_split(X,y,test_size=0.25,random_state=0)

from sklearn.svm import SVC
from sklearn.metrics import accuracy_score
classifier = SVC(kernel="linear") 
classifier.fit(X_train,y_train)

SVC(kernel='linear')

y_pred = classifier.predict(X_test)
accuracy_score(y_test, y_pred)

0.45

When we use linear Kenrnel we got .45 accuracy, Now we will train our model with kernel as poly (if we get low accuracy_score then go for other kernels)

------>Refer notes for clear undestanding<---------

Polynomial Kernal

k(x,y) = (x**T y + c)**d

#we should fin components for poly
df['X1_Square'] = df['X1']**2
df['X2_Square'] = df['X2']**2
df['X1*X2'] = (df['X1']*df['X2'])
df.head()

	X1	X2	Y	X1_Square	X2_Square	X1*x2	X1*X2
0	8.660254	-5.00000	0	75.000000	25.000000	-43.301270	-43.301270
1	8.717792	-4.89899	0	75.999898	24.000102	-42.708375	-42.708375
2	8.773790	-4.79798	0	76.979390	23.020610	-42.096467	-42.096467
3	8.828277	-4.69697	0	77.938476	22.061524	-41.466150	-41.466150
4	8.881281	-4.59596	0	78.877155	21.122845	-40.818009	-40.818009

Independent and dependent variables

X = df[['X1','X2','X1_Square','X2_Square','X1*X2']]
y = df['Y']

X_train, X_test , y_train ,y_test = train_test_split(X,y,test_size=0.25,random_state=0)

For 3D use plotly

!pip install plotly

Collecting plotly
  Downloading plotly-5.5.0-py2.py3-none-any.whl (26.5 MB)
     |████████████████████████████████| 26.5 MB 972 kB/s eta 0:00:01
Collecting tenacity>=6.2.0
  Downloading tenacity-8.0.1-py3-none-any.whl (24 kB)
Requirement already satisfied: six in /Users/harikrishnareddy/opt/anaconda3/lib/python3.8/site-packages (from plotly) (1.15.0)
Installing collected packages: tenacity, plotly
Successfully installed plotly-5.5.0 tenacity-8.0.1
WARNING: You are using pip version 21.2.4; however, version 21.3.1 is available.
You should consider upgrading via the '/Users/harikrishnareddy/opt/anaconda3/bin/python -m pip install --upgrade pip' command.

import plotly.express as px

fig = px.scatter_3d(df,x='X1',y='X2',z='X1*X2',color='Y')
fig.show()

fig = px.scatter_3d(df,x='X1_Square',y='X2_Square',z='X1*X2',color='Y')
fig.show()

classifier = SVC(kernel="linear") 
classifier.fit(X_train,y_train)
y_pred = classifier.predict(X_test)
accuracy_score(y_test, y_pred) 

1.0